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video coding 



(57) A method provides a fast motion search in ad- 
vanced video signal coding systems based on a refer- 
ence-frame prediction and a block-mode prediction so 
that a motion search of each block mode and each ref- 
erence frame is not required. A reference frame predic- 
tion f p , spacedfrom thecurrentframe (22) by "p" number 
of frames, is determined by: p = min(n-1 , p 0 + max(a,b, 
c,d)); wherein p 0 is a pre-chosen positive integer, n is 
the total number of reference frames (12, 14, 16, 18, 
20), wherein A, B, C, and D are image blocks adjacent 
to a searched block, and wherein the reference image 
blocks have been chosen from reference frames f a , f b , 



f c and f d . The search is conducted within frames f 0 to f p , 
which is a subset of all the n reference frames (12, 14, 
16,18, 20), so that the total computational burden is sig- 
nificantly decreased with respect to prior art searches. 
The mode frequency prediction is based on thefrequen- 
cies of the block modes: F° = a • min(F mA , F mB , F mC , 
F mD ); wherein a is a positive parameter less than 1 .0. 
The block-mode selection is then conducted using the 
mode-freq uency prediction. Each mode m among all the 
M possible modes will be considered if F m is greater 
than or equal to F° If F m is less than F° then mode m 
will be skipped during the motion search. 
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Description 

Field Of The Invention 

[0001] This invention relates to a method of conduct- 
ing fast motion searching in advanced video signal cod- 
ing systems and, more particularly, to methods of 
searching using a subset of multiple reference frames 
and/or a subset of multiple image block modes. 

Background Of The Invention 

[0002] A video information format provides visual in- 
formation suitable to activate a television screen, or be 
stored on a video tape. Generally, video data is organ- 
ized in a hierarchical order. A video sequence is divided 
into groups of frames, and each group can be composed 
of a series of single frames. Each frame is roughly equiv- 
alent to a still picture, with the still pictures being updat- 
ed often enough to simulate a presentation of continu- 
ous motion. A frame is further divided into macroblocks. 
In H.26P and MPEG-X standards (Moving Picture Ex- 
perts Group), a macroblock is made up of 1 6x1 6 pixels, 
depending on the video format. A macroblock always 
has an integer number of blocks, such as an 8x8 pixel 
coding unit. 

[0003] Video compression is a critical component for 
any application which requires transmission or storage 
of video data. Compression techniques compensate for 
motion by reusing stored information in previousframes. 
This technique is referred to as temporal redundancy. 
Compression also occurs by transforming data in the 
spatial domain to the frequency domain. 
[0004] Motion compensation is a fundamental tech- 
nique used in video compression such as defined by the 
Moving Picture Experts Group (MPEG) and Internation- 
al Telecommunications Union (ITU) standards. Motion 
estimation is perhaps the most demanding task of a vid- 
eo encoder. Many algorithms and techniques have been 
proposed in the past for conducting fast motion search- 
es. However, these methods apply various fast search 
strategies for certain single block modes, e.g., such as 
an 8x8 block mode, within only one single reference 
frame. None of the prior art methods known to Applicant 
have considered conducting a fast search with multiple 
reference frames and multiple image block modes, 
which is becoming one of the latest techniques in video 
coding. For example, in the ongoing ITU-T H.26L video- 
coding standard, up to seven block modes are consid- 
ered. Moreover, there is no theoretical limit on the 
number of reference frames that may be considered 
during a motion search. These latest techniques im- 
prove video coding efficiency by providing better motion 
compensation. However, these techniques also in- 
crease the computational burden significantly, especial- 
ly for the motion search. 

[0005] In particular, traditional fast motion search 
techniques use a single reference frame and coding 



mode. The reference frame that is used and the coding 
mode are each specified before the search is conduct- 
ed. Direct application of this method to a search of mul- 
tiple reference frames and multiple modes, multiplies 
5 the complexity of the search by the number of reference 
frames and modes. For example, a motion search with 
seven block modes and five reference frames, which is 
atypical configuration in H.26L, requires thirty-five tra- 
ditional motion searches. Even if fast search algorithms 
10 are employed for each of the frame searches, the com- 
plexity is multiplied by thirty-five. 

Summary Of The Invention 

15 [0006] The method of the present invention simplifies 
the motion search by reducing the number of frames and 
modes searched without a significant loss in coding per- 
formance. The invention provides a fast motion search 
method based on a reference-frame prediction and a 

20 block-mode prediction so thatthe motion search of each 
image block is not required to search all of the reference 
frames and all of the block modes. In particular, a refer- 
ence frame prediction fp, spaced from the current frame 
by "p" number of frames, can be determined by: 

25 

p = min(n-1,p 0 + max(a,b,c,d)); 

wherein p 0 is a pre-chosen positive integer (i.e., an ad- 
30 dition factor), n is the total number of reference frames, 
wherein A, B, C and D are image blocks adjacent to 
searched block E, and wherein the reference image 
blocks for image blocks A, B, C and D have been chosen 
from reference frames f a , f b , f c and f d . The search is con- 
35 ducted within frames f 0 to f p , which is a subset of all the 
n reference frames, so that the total computational bur- 
den is significantly decreased with respect to prior art 
searches. 

[0007] For an image block being coded, such as block 
40 E, the block mode selection can be based on the block 
modes in the neighboring blocks, A, B, C and D, which 
have been coded in the modes of mA, mB. mC and mD. 
The frequency of each image block mode F m is the 
number of times the block mode m is used for all the 
45 blocks in the previous w frames and for the blocks in the 
current framethat have been coded. The modefrequen- 
cy prediction is then made based on the frequencies of 
the block modes: 

so 0 

F =a-min(F mA ,F mB; F mC , F mD ); 

wherein a is a positive parameter less than 1 .0 (i.e., a 
multiplication factor). The block-mode selection can 
55 then be conducted using the mode-frequency predic- 
tion. Each mode m among all the M possible modes will 
be considered if F m is greater than or equal to F°. If F m 
is less than F° then that particular mode m will be 



10 



50 



2 



3 



EP 1 241 893 A2 



4 



skipped during the motion search. 
[0008] In particular, the invention comprises: in a dig- 
ital video system where a video sequence is represent- 
ed by a series of frames, including a current frame and 
multiple previous reference frames positioned rear- 
wardly in time with respect to said current frame, each 
separated by a predetermined time interval, the frames 
being divided into a plurality of blocks with predeter- 
mined positions, with each block including a predeter- 
mined matrix of pixel data, a method of efficiently esti- 
mating a change in position of an image represented by 
a matrix of pixel data in an image block in the current 
frame from corresponding matrices of pixel data in a pre- 
vious frame of said series of reference frames, by de- 
termining the location of an optimal reference block with- 
in said series of reference frames, wherein said optimal 
reference block corresponds to said image block, the 
method comprising the steps of: selecting an image 
block in the current frame; selecting a number of refer- 
ence frames; selecting a number of blocks adjacent to 
said image block in the current frame; selecting a value 
for an addition factor; for each of said selected blocks 
adjacent to said image block in the current frame, de- 
termining a reference image block in one of said number 
of reference frames; calculating a subset of frames of 
said number of reference frames in which to search for 
said optimal reference block, wherein said subset of 
frames comprises multiple frames positioned rearward- 
ly in time from said current frame, wherein the calcula- 
tion comprises choosing the minimum of either the 
number of reference frames minus one, or the addition 
factor plus the maximum of the number of frames count- 
ed rearwardly in time from said current frame to reach 
the frame containing the reference image block in said 
one of said number of reference frames for each of said 
reference image blocks; and searching the subset of 
frames for said optimal reference block. 
[0009] The invention furthercomprises in a digital vid- 
eo system where a video sequence is represented by a 
series of frames, including a current frame and multiple 
previous reference frames positioned rearwardly in time 
with respect to said current frame, each separated by a 
predetermined time interval, the frames being divided 
into a plurality of blocks with predetermined positions, 
with each block including a predetermined matrix of pix- 
el data, a method of efficiently estimating a change in 
position of an image represented by a matrix of pixel 
data in an image block in the current frame from corre- 
sponding matrices of pixel data in a previous frame of 
said series of reference frames, by determining the lo- 
cation of an optimal reference block within said series 
of reference frames, wherein said optimal reference 
blockcorrespondsto said image block, the method com- 
prising the steps of: selecting an image block in the cur- 
rent frame; selecting a number of reference frames; se- 
lecting a number of blocks adjacent to said image block 
in the current frame; selecting a number of image block 
modes; determining the mode of each of said selected 



number of blocks adjacent to said image block in the 
current frame; determining the frequency of each image 
block mode within said number of reference frames; se- 
lecting a multiplication factor; calculating a mode-fre- 

5 quency prediction factor by multiplying the multiplication 
factor by the minimum one of the frequency of each im- 
age block mode; calculating a subset of modes of said 
numberof image block modes in which to search for said 
optimal reference block, wherein said subset of modes 

10 comprises each of the modes of said number of image 
block modes when the frequency of each of said modes 
of said number of image block modes is greater than or 
equal to said mode-frequency prediction factor, and 
wherein said subset of modes excludes a particular 

15 mode of said number of image block modes when the 
frequency of the particular mode is less than said mode- 
frequency prediction factor; and searching the subset of 
modes of said number of image block modes for said 
optimal reference block. 

20 [0010] The invention also comprises in a digital video 
system where a video sequence is represented by a se- 
ries of frames, including a current frame and multiple 
previous reference frames positioned rearwardly in time 
with respect to said current frame, each separated by a 

25 predetermined time interval, the frames being divided 
into a plurality of blocks with predetermined positions, 
with each block including a predetermined matrix of pix- 
el data, a method of efficiently estimating a change in 
position of an image represented by a matrix of pixel 

so data in an image block in the current frame from corre- 
sponding matrices of pixel data in a previous frame of 
said series of reference frames, by determining the lo- 
cation of an optimal reference block within said series 
of reference frames, wherein said optimal reference 

35 block corresponds to said image block, the method com- 
prising the steps of: selecting an image block in the cur- 
rent frame; selecting a numberof reference frames; se- 
lecting a number of blocks adjacent to said image block 
in the current frame; selecting a value for an addition 

40 factor; for each of said selected blocks adjacent to said 
image block in the current frame, determining a refer- 
ence image block in one of said number of reference 
frames; calculating a subset of frames of said number 
of reference frames in which to search for said optimal 

45 reference block, wherein said subset of frames compris- 
es multiple frames positioned rearwardly in time from 
said current frame, wherein the calculation comprises 
choosing the minimum of eitherthe number of reference 
frames minus one, or the addition factor plus the maxi- 

50 mum of the number of frames counted rearwardly in time 
from said current frame to reach the frame containing 
the reference image block in said one of said number of 
reference frames for each of said reference image 
blocks; selecting a number of image block modes; de- 

55 termining the mode of each of said selected number of 
blocks adjacent to said image block in the current frame; 
determining the frequency of each image block mode; 
selecting a multiplication factor; calculating a mode-fre- 
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quency prediction factor by multiplying the multiplication 
factor by the minimum one of the frequency of each im- 
age block mode; calculating a subset of modes of said 
number of image block modes in which to search forsaid 
optimal reference block, wherein said subset of modes 
comprises each of the modes of said number of image 
block modes when the frequency of each of said modes 
of said number of image block modes is greater than or 
equal to said mode-frequency prediction factor, and 
wherein said subset of modes excludes a particular 
mode of said number of image block modes when the 
frequency of the particular mode is less than said mode- 
frequency prediction factor; and searching the subset of 
frames of said number of reference frames and search- 
ing the subset of modes of said number of image block 
modes for said optimal reference block. 
[0011] Accordingly, an object of the invention is to pro- 
vide a method of conducting a fast motion search in ad- 
vanced video coding. 

[0012] Another object of the invention is to provide a 
method of conducting reference frame prediction and/ 
or block mode prediction in a fast motion search. 
[0013] A further object of the invention is to provide a 
method of conducting a search including a subset of 
multiple reference frames and/or a subset of multiple im- 
age block modes. 

Brief Description Of The Drawings 

[0014] 

FIG. 1 is a schematic of multiple reference frames 

utilized during a motion search. 

FIG. 2 is a schematic of an image block in a current 

image frame and its adjacent image blocks. 

FIG. 3 is a flowchart of the method of reference 

frame prediction. 

FIG. 4 is a flowchart of the method of block mode 
prediction. 

FIG. 5 is a graph of experimental results of a relative 

motion search for the QCI F format. 

FIG. 6 is a graph of the rate-distortion performance 

of the coding system for the QCIF sequence of FIG. 

5. 

FIG. 7 is a graph of experimental results of a relative 
motion search for the CIF format. 
FIG. 8 is a graph of the rate-distortion performance 
of the coding system for the CIF sequence of FIG. 7. 

Detailed Description Of The Preferred Embodiments 

[0015] Turning now to the drawings, FIG. 1 shows a 
schematic diagram 1 0 of multiple reference frames 12, 
14, 16, 18. 20, and 22. The frames are arranged along 
a horizontal axis 24 representing the flow of time, with 
frame 22 being the current frame and frame 12 being 
the frame furthest in the past, i.e., most rearwardly in 
time from frame 22. When integer n (with n>1) reference 



frames are used in the motion search for an image block 
24 within current frame 22, the probability of selecting 
the optimal reference block from a reference frame tem- 
porally close to the current frame, for example frame 20 , 
5 labeled as frame f 0 , is much higher than the probability 
of choosing the reference block from a reference frame 
far away from the current frame such as frame 12, la- 
beled as frame \ n .^ . Therefore, while consuming a great 
deal of computational time, a search through all of the 
10 n reference frames for all the image blocks in the current 
frame 22 will not give a significant benefit. To take ad- 
vantage of this understanding, a reference-frame pre- 
diction technique has been developed. 
[0016] FIG. 2 shows several blocks within a single 
15 frame. Block 26, labeled E, represents an image block 
in the current image frame for which a motion search 
will be conducted. Blocks 28, 30, 32 and 34, labeled A, 
B, C and D, respectively, are the possible image blocks 
adjacent to block 26, i.e., block E, in the current frame 
20 whose reference image blocks have been chosen from 
reference frames f a , f b , f c and f d , respectively, wherein 
f a is the reference frame within the sequence of n frames 
containing the reference image block for block A. A pre- 
diction frame, f p , can be derived wherein p is the number 
25 of frames from the current frame, with p defined as the 
following: 

Equation 1 : p = min(n-1 , p 0 + max(a ; b,c,d)); 

30 

where p 0 is a pre-chosen positive integer parameter, i. 
e., the addition factor. The motion search for image 
block 26, i.e., image block E ; can be conducted within 
frames f 0 to f p , which is a subset of all of the n reference 
35 frames, so that the total computational burden is signif- 
icantly decreased. The addition factor p 0 can be 
changed or adjusted during the video coding process to 
control the motion search and the video quality, i.e., p 0 
can be changed dynamically during the coding process. 
40 [0017] The prediction of f p can be very flexible. In par- 
ticular, the prediction can be made from various sets of 
the neighboring image blocks. For example, the follow- 
ing equations will give very similar results: 

45 

Equation 2: p = min(n-1 , p 0 + max(a,b,c)); 
or 

50 

Equation 3: p = min(n-1 , p 0 + max(a,b)). 

[0018] In a special case, where all the neighboring 
blocks used for prediction are out of the image frame or 
55 are coded as intra blocks, the value of p will be set to 
(n-1), which means all of the reference frames will be 
considered during the motion search. Accordingly, as a 
worst case scenario, in particular search scenarios, the 
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method of the present invention will require the same 
number of computations as the prior art full searches. 
However, as stated above, the method of the present 
invention, due to the frame prediction computation, typ- 
ically will comprise a search of only a subset of the full 
number of frames, thereby reducing the computational 
burden of the search when compared to prior art search 
methods. 

[0019] FIG. 3 shows a flowchart of the method of ref- 
erence frame prediction. In particular, step 36 comprises 
choosing the image blocksize, such as a size of 16x16. 
Step 38 comprises choosing n, the number of reference 
frames. Step 40 comprises choosing the possible image 
blocks adjacent to the block to be searched, such as 
choosing blocks A and B, choosing blocks A, B and C, 
choosing blocks A, B, C and D, choosing blocks A, B, 
C, D and F. etc. Step 42 comprises choosing p 0 , the pre- 
determined positive integer. Step 44 comprises calcu- 
lating p by Equations 1 , 2, 3 or other similar equation for 
the chosen number of blocks adjacent to the block to be 
searched (determined in step 40). Step 46 comprises 
searching frames f 0 to f p , wherein p is determined in step 
44. Those skilled in the art will understand that the indi- 
vidual steps shown in FIG. 3 may be conducted in a va- 
riety of different sequences, i.e., the initially chosen var- 
iables may be chosen in any order. Moreover, the search 
may be conducted before, after or simultaneous to a 
block mode prediction search as shown in FIG. 4. 
[0020] Similar to the observations discussed above 
with respect to multiple reference frames, Applicant has 
observed that when there are M possible image block 
modes, the determination of the mode for each image 
block during the motion search is also very costly if each 
of the M modes are tested. However, it has been found 
in experiments conducted by Applicant that most of the 
image blocks are actually coded using only a subset of 
the M modes. The subset of the modes may vary de- 
pending on the specific video content and'coding pa- 
rameters. To make better use of this understanding, a 
block-mode prediction method has been developed. 
[0021] For an image block being coded, such as block 
26 in FIG. 2, i.e., block E, the block mode selection can 
be based on the block modes of the neighboring blocks 
28, 30, 32 and 34, i.e., blocks A, B, C and D, respective- 
ly, which have been coded in the modes of mA. mB, mC 
and mD, respectively. First, a concept of block-modefre- 
quency is defined using a temporal window across w 
frames, wherein w may or may not be equal to n, the 
number of frames chosen in the steps of FIG. 3. The 
frequency of each image block mode Fm is the number 
of times the block mode m is used for all the blocks in 
the previous w frames and for the blocks in the current 
frame that have been coded. A mode-frequency predic- 
tion, F°, can then be made based on the frequencies of 
the block modes: 

Equation 4: F° = a ■ min(F mA , F mB , F mC; F mD ); 



where a is a positive parameter less than 1 . The multi- 
plication factor a can be changed or adjusted during the 
video coding process to control the motion search and 
the video quality, i.e., a can be changed dynamically 
5 during the coding process 

[0022] The block mode selection can then be con- 
ducted using the mode-frequency prediction. Each 
mode m among all the M possible modes will be consid- 
ered if Equation 5 is true: 

10 

Equation 5: F m > F°. 

[0023] If Equation 6 is true, then mode m will be 
15 skipped during the motion search: 

Equation 6: F m < F°. 

20 [0024] The mode-frequency prediction F° can be very 
flexible. The prediction can be made from various sets 
of the neighboring image blocks. For example, the fol- 
lowing equations will give very similar results: 

25 0 

Equation 7: F = a • min(F mA , F mB , F mC ); 

or 

30 _ o 

Equation 8: F = a-min(F mA , F mB ). 

[0025] In a special case, when all the neighboring 
blocks 28, 30, 32 and 34, i.e., blocks A, B, C and D, 

35 respectively, are out of the image frame orcoded as intra 
blocks, the mode-frequency prediction is set to zero, 
which means that all of the M block modes will be con- 
sidered during each search. In another special case, all 
the M possible modes are considered when the first w 1 

40 frames of a sequence are coded while the system ac- 
cumulates statistics for the modes. The parameter w 1 
typically is a positive integer not larger than w, the total 
number of frames. 

[0026] FIG. 4 is a flowchart of the method of block 
45 mode prediction. In particular, step 51 comprises choos- 
ing M, the number of possible image block modes. Step 
52 comprises choosing the possible image blocks adja- 
cent to the block to be searched, such as choosing 
blocks A and B, choosing blocks A, B and C, choosing 
50 blocks A, B, C and D, choosing blocks A, B, C, D and F, 
etc. Step 54 comprises determining mA, mB, mC, mD, 
etc, which are the block modes of each of the chosen 
adjacent blocks. Step 56 comprises choosing w, the to- 
tal number of frames. Step 58 comprises choosing w 1; 
55 the subset of frames of w for the special search case 
set forth above. Step 60 comprises determining F mA , 
F mB , F mC , F mD , etc, which are the frequencies of each 
image block, i.e., the number of times block mode m is 
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used for all the blocks in the previous w frames and for 
the blocks in the current frame that have been coded. 
Step 62 comprises choosing a, a positive parameter 
less than one. Step 64 comprises determining F° by 
Equations4, 7, 8orothersimilarequationforthechosen 
number of blocks adjacent to the block to be searched 
(determined in step 52). Step 66 comprises determining 
whether F m is greater than or equal to F°. If so, in step 
68 each mode m among the M possible modes is 
searched. If not, in step 70 mode m is skipped during 
the motion search. Those skilled in the art will under- 
stand that the individual steps shown in FIG. 4 may be 
conducted a variety of different sequences, i.e., the ini- 
tially chosen variables may be chosen in any order. 
Moreover, the method of conducting a combined frame 
prediction search and block mode prediction search 
simply entails combining in one search the steps set 
forth in both of FIGS. 3 and 4. 
[0027] FIG. 5 is a graph of the experimental results of 
a relative motion search for the quarter common inter- 
mediate format (QCIF) format. In this particular experi- 
ment the following parameters were used: the image 
block size was 1 6x1 6; n = 5; p 0 = 1 , m = 7 (which cor- 
responds to block tiling patterns of 4x4, 4x8, 8x4, 8x8, 
8x1 6. 1 6x8, and 1 6x1 6); a = 1/2; w = 4; and w 1 = 1 . The 
experimental results shown are for a QCIF format se- 
quence at 10 frames per second using frame prediction, 
mode prediction, and a combination of frame and mode 
prediction, respectively. The speed up ratio using the 
combined frame and mode prediction search is in the 
range of three to five times faster than the traditional full 
search of all frames and modes, i.e., 35% to 20% of the 
time required for the full search, depending on the quan- 
tization parameter utilized, as represented by the hori- 
zontal axis in FIG. 5. 

[0028] The quantization parameter (QP) is a param- 
eter used in image and video coding to control the cod- 
ing quality. Usually, a high QP provides low-quality im- 
ages and a low QP delivers high-quality images l as un- 
derstood by those skilled in the art. Therefore, video 
coding is sensitive to the accuracy of motion estimation 
when the QP is relatively low. This explains why the dis- 
closed method has different performance with respect 
to the different QP values. Accordingly, the H.26L stand- 
ard is used as only one example to demonstrate the per- 
formance of the disclosed invention. 
[0029] FIG. 6 is a graph of the rate-distortion perform- 
ance of the coding system for the QCIF sequence 
shown in FIG. 5. The horizontal axis represents the bit 
rate, and the vertical axis represents the distortion, i.e., 
the peak signal to noise ratio (PSNR) of the image lu- 
minance. Comparing the frame prediction search, the 
mode prediction search, and the combined frame and 
mode prediction search to the conventional full search 
of all frames and all modes, shows that there is little deg- 
radation in performance when utilizingthe method of the 
present invention. In particular, the distortion introduced 
by the method of the present invention, which utilizes a 



subset of the reference frames and a subset of the im- 
age block modes, is less than 0.5 dB in terms of the peak 
signal to noise ratio for a variety of bit rates. 
[0030] FIG. 7 is a graph of experimental results of a 

5 relative motion search for a common intermediate for- 
mat (CIF) format at 30 frames per second using frame 
prediction, mode prediction and a combination of frame 
and mode prediction, respectively. The speed up ratio 
using the combined frame and mode prediction search 

10 is in the range of four to six times faster than the tradi- 
tional full search of all frames and all modes, i.e., 28% 
to 15% of the time required forthefull search, depending 
on the quantization parameter utilized, as represented 
by the horizontal axis in FIG. 7. 

15 [0031] FIG. 8 is a graph of the rate-distortion perform- 
ance of the coding system for the CIF sequence shown 
in FIG. 7. The horizontal axis represents the bit rate, and 
the vertical axis represents the PSNR of image lumi- 
nance. Comparing the frame prediction search, the 

20 mode prediction search, and the combined frame and 
mode prediction search to the conventional full search 
of all frames and all modes, shows that there is little deg- 
radation in performance when utilizing the method of the 
present invention. 

25 [0032] In the example shown, the mode corresponds 
to one of the block sizes 1 6x1 6, 1 6x8, 8x1 6, 8x8, 8x4, 
4x8 and 4x4. However, the present method can be used 
for other block sizes or coding modes. For example, dif- 
ferent coding modes may be used in intra coding where 

so different directions of spatial prediction are used in each 
mode. Moreover, the motion search method within each 
particular reference frame and for one specific block 
mode is not specified for this invention because any of 
a variety of fast search methods may be used within 

35 these stages, as will be understood by those skilled in 
the art. 

[0033] As an example, motion searches using multi- 
ple reference frames and modes have been addressed 
by using a series of independent searches, one per 

40 frame and mode combination. As previously described, 
the methods of reference frame prediction and mode 
prediction reduce the complexity of the motion search 
by selectively reducing the number of ordinary motion 
searches conducted. This benefit applies regardless of 

45 the motion search technique used forthe individual mo- 
tion searches. One particular existing method for fast 
motion searches of a single reference frame and coding 
mode operates by giving a preferred order to the set of 
motion vectors tested, i.e., a spiral search. The success 

50 of these fast searches relies upon finding near optimal 
solutions among the first few vectors tested. This allows 
sub-optimal vectors to be eliminated quickly from later 
parts of the search, giving a computation gain. The ideas 
of reference frame prediction and mode prediction can 

55 enhance this operation. Even if all reference frames and 
coding modes are examined, the reference frame pre- 
diction and mode prediction methods of the present in- 
vention can serve to order the reference frames and the 
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coding modes so that a fast search has a greater prob- 
ability of encountering near optimal vectors in the first 
few reference frames and coding modes of the search. 
In other words, the subset of frames and the subset of 
modes predicted by the calculations set forth herein can 
be searched first, and the remaining frames and modes 
not included in the subset determined by the inventive 
search methods, can be searched second. This order- 
ing of the subsets to be searched first enhances the per- 
formance of such as fast motion search algorithms even 
if all the reference frames and coding modes are exam- 
ined. Thus, the present invention provides a method of 
ordering the reference frames and the modes searched 
by existing fast motion searches so as to enhance the 
fast motion search. 

[0034] Thus, a method of conducting a reference 
frame prediction search, a method of conducting a block 
mode prediction search, and a method of conducting a 
combined frame prediction and block mode prediction 
search, for fast motion searches in advanced video cod- 
ing, have been disclosed. Although preferred methods 
of conducting the searches have been disclosed, it 
should be appreciated that further variations and modi- 
fications may be made thereto without departing from 
the scope of the invention as defined in the appended 
claims. 



Claims 

1. In a digital video system where a video sequence 
is represented by a series of frames (10), including 
a current frame (22) and multiple previous refer- 
ence frames (12, 14, 16, 18, 20) positioned rear- 
wardly in time with respect to said current frame 
(22), each separated by a predetermined time inter- 
val, the frames being divided into a plurality of 
blocks with predetermined positions, with each 
block including a predetermined matrix of pixel da- 
ta, a method of efficiently estimating a change in 
position of an image represented by a matrix of pixel 
data in an image block in thecurrent frame (22) from 
corresponding matrices of pixel data in a previous 
frame of said series of reference frames (12,14,16, 
18, 20), by determining the location of an optimal 
reference block within said series of reference 
frames (12, 14, 16, 18, 20), wherein said optimal 
reference block corresponds to said image block, 
the method comprising the steps of: 

selecting an image block in the current frame 

(22); 

selecting a number of reference frames (12, 1 4, 
16, 18, 20); 

selecting a number of blocks (28, 30, 32, 34) 
adjacent to said image block (26) in the current 
frame (22); 

selecting a value for an addition factor; 



for each of said selected blocks (28, 30, 32, 34) 
adjacent to said image block (26) in the current 
frame (22), determining a reference image 
block in one of said number of referencef rames 

5 (12, 14, 16, 18, 20); 

calculating a subset of frames of said number 
of reference frames (12, 14, 16, 18, 20) in which 
to search for said optimal reference block, 
wherein said subset of frames comprises mul- 

10 tiple frames positioned rearwardly in time from 

said current frame (22), wherein the calculation 
comprises choosing the minimum of either the 
number of reference frames (12, 14, 16, 18, 20) 
minus one, or the addition factor plus the max- 

15 imum of the number of frames counted rear- 

wardly in time from said current frame (22) to 
reach theframe containing the reference image 
block in said one of said number of reference 
frames (12, 14, 16, 18, 20) for each of said ref- 

20 erence image blocks; and 

searching the subset of frames for said optimal 
reference block. 

2. The method of claim 1 wherein the number of blocks 
25 (28, 30, 32, 34) adjacent to said image block (26) in 

the current frame (22) is selected to be less than 
nine. 

3. The method of claim 1 wherein the number of blocks 
30 (28, 30, 32, 34) adjacent to said image block (26) in 

the current frame (22) is selected to be less than 
five. 

4. The method of claim 1 wherein the addition factor 
35 is a positive integer. 

5. The method of claim 1 further comprising the steps 
of: 

40 selecting a number of image block modes; 

determining the mode of each of said selected 
number of blocks (28, 30, 32, 34) adjacent to 
said image block (26) in the current frame (22); 
determining the frequency of each image block 

45 mode; 

selecting a multiplication factor; 
calculating a mode-frequency prediction factor 
by multiplying the multiplication factor by the 
minimum one of the frequency of each image 

50 block mode; 

calculating a subset of modes of said number 
of image block modes in which to search for 
said optimal reference block, wherein said sub- 
set of modes comprises each of the modes of 

55 said number of image block modes when the 

frequency of each of said modes of said 
number of image block modes is greater than 
or equal to said mode-frequency prediction fac- 
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tor, and wherein said subset of modes excludes 
a particular mode of said number of image 
block modes when the frequency of the partic- 
ular mode is less than said mode-frequency 
prediction factor; and 

searching the subset of modes for said optimal 
reference block. 

6. The method of claim 5 wherein the number of image 
block modes selected is seven. 

7. The method of claim 1 wherein said method is con- 
ducted using a format sequence chosen from the 
group consisting of a GIF format sequence and a 
QCIF format sequence. 

8. The method of claim 1 wherein said method is con- 
ducted at a frame speed greater than one frame per 
second and less than sixty frames per second. 

9. The method of claim 5 wherein said method is con- 
ducted at least three times faster than a search in- 
cluding each frame of said number of reference 
frames (12, 14, 16, 18, 20) and each mode of said 
number of image block modes. 

10. The method of claim 1 wherein said method is con- 
ducted at a bit rate of greater than 10,000 and less 
than 250,000. 

11. The method of claim 1 wherein a distortion intro- 
duced to the video sequence is less than 0.5 dBwith 
respect to a peak signal to noise ratio. 

12. The method of claim 1 wherein said method is con- 
ducted in less than fifty two percent of the time re- 
quired to conduct a search including each frame of 
said number of reference frames (12, 14, 16, 18, 
20). 

13. In a digital video system where a video sequence 
is represented by a series of frames (10), including 
a current frame (22) and multiple previous refer- 
ence frames (12, 14, 16, 18, 20) positioned rear- 
wardly in time with respect to said current frame 
(22), each separated by a predetermined time inter- 
val, the frames being divided into a plurality of 
blocks with predetermined positions, with each 
block including a predetermined matrix of pixel da- 
ta, a method of efficiently estimating a change in 
position of an image represented by a matrix of pixel 
data in an image block in thecurrent frame (22) from 
corresponding matrices of pixel data in a previous 
frame of said series of reference frames (12, 14,16, 
18, 20), by determining the location of an optimal 
reference block within said series of reference 
frames (12, 14, 16, 18, 20), wherein said optimal 
reference block corresponds to said image block, 



the method comprising the steps of: 

selecting an image block in the current frame 
(22); 

5 selecting a number of reference frames (12, 14, 

16, 18, 20); 

selecting a number of blocks (28, 30, 32 : 34) 
adjacent to said image block (26) in the current 
frame (22); 

10 selecting a number of image block modes; 

determining the mode of each of said selected 
number of blocks (28, 30, 32, 34) adjacent to 
said image block (26) in the current frame (22); 
determining the frequency of each image block 

15 mode within said number of reference frames; 1 

selecting a multiplication factor; 
calculating a mode-frequency prediction factor 
by multiplying the multiplication factor by the 
minimum one of the frequency of each image 

20 block mode; 

calculating a subset of modes of said number 
of image block modes in which to search for 
said optimal reference block, wherein said sub- 
set of modes comprises each of the modes of 

25 said number of image block modes when the 

frequency of each of said modes of said 
number of image block modes is greater than 
or equal to said mode-frequency prediction fac- 
tor, and wherein said subset of modes excludes 

so a particular mode of said number of image 

block modes when the frequency of the partic- 
ular mode is less than said mode-frequency 
prediction factor; and 

searching the subset of modes of said number 
35 of image block modes for said optimal refer- 

ence block. 

14. The method of claim 1 3 wherein said method is con- 
ducted in less than seventy five percent of the time 

40 required to conduct a search including each mode 
of said number of image block modes. 

15. The method of claim 13 further comprising the steps 
of: 

45 

selecting a value for an addition factor; 
for each of said selected blocks (28, 30, 32, 34) 
adjacent to said image block (26) in the current 
frame (22), determining a reference image 
50 block in one of said number of referencef rames 

(12, 14, 16, 18, 20); 

calculating a subset of frames of said number 
of reference frames (12, 14, 16, 18, 20) in which 
to search for said optimal reference block, 
55 wherein said subset of frames comprises mul- 

tiple frames positioned rearwardly in time from 
said current frame (22), wherein the calculation 
comprises choosing the minimum of either the 
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number of reference frames (1 2,14,16,1 8, 20) 
minus one, or the addition factor plus the max- 
imum of the number of frames counted rear- 
wardly in time from said current frame (22) to 
reach the frame containing the reference image 
block in said one of said number of reference 
frames (12, 14, 16, 18, 20) for each of said ref- 
erence image blocks; and 
searching the subset of frames for said optimal 
reference block. 

16. The method of claim 15 wherein the addition factor 
is one. 

17. The method of claim 13 wherein the multiplication 
factor is one half. 

18. In a digital video system where a video sequence 
is represented by a series of frames (10), including 
a current frame (22) and multiple previous refer- 
ence frames (12, 14, 16, 18, 20) positioned rear- 
wardly in time with respect to said current frame 
(22), each separated by a predetermined time inter- 
val, the frames being divided into a plurality of 
blocks with predetermined positions, with each 
block including a predetermined matrix of pixel da- 
ta, a method of efficiently estimating a change in 
position of an image represented by a matrix of pixel 
data in an image block in thecurrent frame (22) from 
corresponding matrices of pixel data in a previous 
frame of said series of reference frames (12, 14,16, 
18, 20), by determining the location of an optimal 
reference block within said series of reference 
frames (12, 14, 16, 18, 20), wherein said optimal 
reference block corresponds to said image block, 
the method comprising the steps of: 

selecting an image block in the current frame 

(22); 

selecting a number of reference frames (12, 1 4, 
16, 18, 20); 

selecting a number of blocks (28, 30, 32, 34) 
adjacent to said image block (26) in the current 
frame (22); 

selecting a value for an addition factor; 

for each of said selected blocks (28, 30, 32, 34) 

adjacent to said image block (26) in the current 

frame (22), determining a reference image 

block in one of said number of reference frames 

(12,14,16,18,20); 

calculating a subset of frames of said number 
of reference frames (12. 14. 16, 18, 20) in which 
to search for said optimal reference block, 
wherein said subset of frames comprises mul- 
tiple frames positioned rearwardly in time from 
said currentframe (22), wherein the calculation 
comprises choosing the minimum of either the 
number of reference frames (1 2,14,16,1 8, 20) 



minus one, or the addition factor plus the max- 
imum of the number of frames counted rear- 
wardly in time from said current frame (22) to 
reach theframe containing the reference image 
5 block in said one of said number of reference 

frames (12, 14, 16, 18, 20) for each of said ref- 
erence image blocks; 

selecting a number of image block modes; 
determining the mode of each of said selected 
10 number of blocks (28, 30, 32, 34) adjacent to 

said image block (26) in the currentframe (22); 
determining the frequency of each image block 
mode; 

selecting a multiplication factor; 
15 calculating a mode-frequency prediction factor 

by multiplying the multiplication factor by the 
minimum one of the frequency of each image 
block mode; 

calculating a subset of modes of said number 
20 of image block modes in which to search for 

said optimal reference block, wherein said sub- 
set of modes comprises each of the modes of 
said number of image block modes when the 
frequency of each of said modes of said 
25 number of image block modes is greater than 

or equal to said mode-frequency prediction fac- 
tor, and wherein said subset of modes excludes 
a particular mode of said number of image 
block modes when the frequency of the partic- 
30 ular mode is less than said mode-frequency 

prediction factor; and 

searching the subset of frames of said number 
of reference frames (12, 14, 16, 18, 20) and 
searching the subset of modes of said number 
35 of image block modes for said optimal refer- 

ence block. 

19. The method of claim 1 wherein the addition factor 
is adjusted dynamically. 

40 

20. The method of claim 13 wherein the multiplication 
factor is adjusted dynamically. 

21. The method of claim 1 wherein said subset of 
45 frames is searched for said optimal reference block 

and thereafter a remainder of said number of refer- 
ence frames (12, 14, 16, 18, 20) are searched for 
said optimal reference block, such that said subset 
of frames is given a preferential order during said 
50 search. 

22. The method of claim 13 wherein said subset of 
modes of said number of image block modes is 
searched for said optimal reference block and 

55 thereafter a remainder of said number of image 
block modes is searched for said optimal reference 
block, such that said subset of modes is given a 
preferential order during said search. 
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